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PREFACE 


CUSTOMER  SATISFACTION  REPORTING 
CONDUCTED  BY  THE  DEFENSE  TECHNICAL  INFORMATION  CENTER 

The  Defense  Technical  Information  Center  (DTIC)  has  surveyed  its  registered  customers  since 
FY  1999.  There  are  two  surveys  conducted  yearly: 

Customer  Satisfaction  Survey  A  complete  sampling  (census)  method  of  all  users  was  used 
excluding  the  following:  DTIC  employees,  Air  Force  Materiel  Command  (AFMC)  personnel, 
Small  Business  Innovative  Research  (SBIR),  foreign  or  overseas  users  and  users  identified  as 
Top  200.  Surveys  were  conducted  in  FY  1999  through  FY  2003. 

Top  200  Users  Roughly  200  users  defined  in  terms  of  dollar  value  of  billing,  number  of 
documents  ordered,  and  downloads  from  Secure  STINET.  All  200+  users  were  surveyed. 
Surveys  were  conducted  in  FY  2000  through  FY  2003. 

A  combined  Annual  Survey  Composite  Data  Report  was  developed  in  FY  2001  and  FY  2002. 
An  analysis  highlighting  differences  between  the  two  surveys  is  planned  for  FY  2003. 

The  DTIC  Performance  Report,  sent  to  the  Defense  Information  Systems  Agency  (DISA), 
contains  a  report  on  overall  customer  satisfaction.  This  is  derived  from  the  Annual  Survey 
Composite  Data  Report  and  reflects  data  from  both  the  Top  200  Users  Survey  and  the  Customer 
Satisfaction  (CS)  survey. 


Customer  Satisfaction  Survey  Results:  How  Do  We  Measure  Up? 


Based  upon  the  CS  survey  findings,  the  majority  of  the  2003  survey  respondents  were  satisfied 
with  DTIC  services  as  a  whole.  While  the  result  is  generally  positive,  the  score  did  decline  4 
points  since  the  2002  survey  (down  from  81  percent  to  77  percent  ).  However,  DTIC  continues 
to  exceed  the  Federal  Government  American  Customer  Satisfaction  Index  (ACSI) 
baseline/benchmark  score.  The  ACSI  survey  was  conducted  in  November  2002,  while  the  DTIC 
CS  Survey  was  conducted  in  July  2003.  The  results  are  illustrated  on  the  graph  below. 


Customer  Service 
Elements 

FY  2001 

CS  Survey 

FY  2002 

CS  Survey 

FY  2003 

CS  Survey 

DTIC  Composite 
Scores 

76% 

81% 

77% 

ACSI  Federal 
Government 
Benchmark* 

68.6% 

71.1% 

70.2% 

DTIC  Excels  by 

+7.4 

+9.9 

6.8% 

*ACSI  is  the  official  service  quality  benchmark  for  the  Federal  Government 

Note:  Data  relevant  to  this  chart  is  detailed  in  Part  II 
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2003  DTIC  Customer  Satisfaction  Data  Summary 
Part  I:  Introduction 

Purpose:  This  report  contains  the  findings  of  the  2003  Defense  Technical  Infonnation  Center 
(DTIC)  Customer  Satisfaction  (CS)  survey  conducted  in  July  2003.  The  purpose  of  the  survey 
was  to  help  DTIC  (1)  gauge  the  level  of  satisfaction  among  its  general  users  and  (2)  identify 
possible  areas  for  improving  its  products  and  services.  This  report  describes  the  survey  process, 
presents  its  results,  and  recommends  actions  identified  by  survey  respondents.  Specifically,  this 
report  summarizes  the  results  of  the  2003  CS  survey.  It  also  compares  findings  from  the  2001 
and  2002  CS  user  data.  Collectively,  the  report  allows  us  to  work  toward  the  following 
objectives: 

■  To  improve  customer  retention 

■  To  detennine  the  perceived  quality  of  products,  services  and  customer  care 

■  To  indicate  trends  in  products,  services  and  customer  care 

■  To  benchmark  our  customer  satisfaction  results  with  other  federal  government  agencies 

To  ensure  that  we  gained  fundamental  knowledge  objectives  from  our  users,  the  2003  CS  survey 
was  specifically  designed  by  querying  our  registered  users  in  five  (5)  categories  listed  below: 

■  Customer  Service  Experiences 

■  Global  Customer  Service  Performance 

■  DTIC  Offerings 

■  User  Demographics 

■  Communication/Access  and  Infonnation  Requirements 

With  regard  to  these  five  (5)  categories,  users  were  asked  to  answer  questions  geared  toward 
customer  service  factors:  accessibility,  accuracy,  courtesy,  helpfulness,  knowledge, 
professionalism,  responsiveness,  and  speed  of  service.  Other  questions  targeted  aspects  of 
DTIC's  products  (i.e.,  customer  involvement,  quality  and  access)  as  well  as  adequate,  useful,  and 
prompt  information  distribution. 

Methodology:  Web-based  and  e-mail  surveys  were  the  primary  collection  methods  selected  for 
this  effort.  One-on-one  telephone  interviews  were  used  to  gather  contact  infonnation  and,  on 
occasion,  to  administer  the  survey.  These  multiple  collection  methods  were  selected  not  only  to 
offer  our  users  a  variety  of  survey  response  options,  but  also  to  increase  response  rates. 

The  target  population  for  this  effort  was  2680  registered  users.  Active  users  not  surveyed 
included  DTIC  employees,  Air  Force  Materiel  Command  (AFMC)  personnel,  Small  Business 
Innovative  Research  (SBIR),  foreign  or  overseas  users  and  users  identified  as  Top  200.  A 
complete  sampling  (census)  method  was  chosen  based  upon  (1)  the  potential  to  increase  response 
rates  and  (2)  knowing  that  anomalies  can  occur  and  have  occurred  when  launching  the  DTIC 
survey  contact  user  list.  After  three  (3)  e-mail  contact  attempts,  73 1  users  responded  to  the 
survey.  There  were  824  non-deliverable  emailed  surveys.  Statistically,  the  survey  results  are 
reliable.  This  was  achieved  with  a  confidence  level  of  95  percent  and  a  7  percent  error  tolerance 
with  an  overall  response  rate  of  27  percent.  The  response  rate  for  the  2003  CS  survey  increased 
12  percentage  points  in  comparison  to  last  year's  response  rate,  which  was  15  percent. 


Possible  reasons  73  percent  of  the  2003  CS  survey  population  did  not  respond  to  the  survey 
could  be: 

•  Internal  DTIC  problems  with  Microsoft  Outlook  so  user  never  received  survey 

•  User  could  not  be  contacted  due  to  invalid  contact  infonnation  or  user  no  longer  with 
organization 

•  User  would  not  answer  survey  (too  busy,  does  not  complete  surveys,  not  familiar  enough 
with  DTIC  to  answer) 

Interpretation  of  the  Data: 

When  considering  the  results  of  Part  I  of  the  survey  process,  it  is  important  to  recognize  not 
only  the  characteristics  of  the  customers  who  could  not  be  surveyed  due  to  lack  of  valid  contact 
information,  but  also  the  42  percent  of  general  registered  users  who  either  did  not  get  the  survey 
or  who  did  not  respond.  Analysis  of  the  survey  results  will  always  need  to  be  tempered  by 
considerations  of  who  did  or  did  not  respond,  and  to  what  extent  those  who  responded  have  the 
same  characteristics  or  views  as  those  who  did  not  respond. 

Overall  favorable  percentages  are  based  on  normalized  data,  a  process  which  uses  the  total 
number  of  valid  responses  along  a  five  point  scale  and  converts  these  to  a  100  point  scale.  These 
may  vary  somewhat  from  the  total  percentage  of  favorable  responses.  The  overall  favorable 
percentages  are  derived  from  the  “Mean  (normalized)”  column  in  the  tables  for  quantitative 
results  at  Appendix  B.  Most  analytic  conclusions  throughout  this  report  are  derived  from  these 
"overall  percentages." 
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Overall  Findings 


(Data  relevant  to  the  overall  findings  below  are  detailed  in  Appendix  B) 

Customer  Service  Issues 

■  Seventy-seven  percent  of  surveyed  respondents  reported  that  they  were  satisfied  with  the 
level  of  DTIC's  customer  care,  (reference  page  36) 

DTIC  Offerings 

DTIC  Online  Services 

■  Sixty-nine  percent  of  respondents  rated  DTIC  Online  Services  quality  as  “Excellent”  to 
“Very  Good.”  (reference  page  37) 

■  Private  STINET  was  the  most  widely  used  DTIC  Online  Service  when  the  survey  was 
conducted  in  July  2003.  (reference  page  9) 

DTIC  Homepage 

■  CS  survey  respondents  rated  the  overall  satisfaction  with  the  DTIC  Homepage  at  69  percent. 
(reference  page  38) 

■  Accessibility,  appearance,  and  content  were  the  top  rated  quality  factors  for  the  DTIC 
Homepage,  (reference  pages  38) 

■  Of  the  top  three  user  groups  responding  to  the  survey,  Librarians  rated  the  overall  quality  of 
the  DTIC  Homepage  the  highest  at  7 1  percent,  (reference  page  38) 

■  Seventy-five  percent  of  users  responding  to  the  survey  reported  using  the  DTIC  Homepage  in 
the  past  12  months,  (reference page  37) 

DTIC  Annual  Users  Conference 

■  Only  12  percent  of  CS  survey  respondents  reported  attending  a  DTIC  Annual  Users 
Conference  in  the  past  3  years,  (reference  page  39) 

User  Demographics 

■  The  majority  of  respondents  were  DoD  employees,  (reference  page  39) 

■  The  majority  of  the  users  responding  came  from  three  user  groups:  Librarians,  Engineers  and 
Researcher/ Analysts,  (reference  page  40) 

■  The  vast  majority  of  survey  respondents  reported  that  they  primarily  act  as  information 
providers  to  others,  (reference  page  40) 
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■  The  largest  number  of  respondents  reported  being  registered  with  DTIC  for  1-2  years. 
(reference  page  40) 

Communications,  Access  and  Information  Requirements 

■  The  vast  majority  of  users  responding  to  the  survey  find  that  their  infonnational  needs  are 
being  met  by  searching  DTIC's  collection,  (reference  page  40) 

■  CS  survey  respondents  find  the  Internet/Online  services  as  the  most  often  used 
communication  channel  to  contact  DTIC.  (reference page  41) 

■  Surveyed  users  reported  they  are  satisfied  that  DTIC  does  an  adequate  job  in  communicating 
information  about  new  changes  in  and/or  enhanced  products  and  services  offerings,  training 
opportunities,  etc.  (reference  41-42) 

■  Over  half  of  the  users  responding  reported  the  Internet/Online  Services  as  the  preferred 
medium  of  ordering  documents,  (reference  43) 

General  Findings 

■  Eighty-five  percent  of  all  survey  respondents  indicated  that  DTIC  is  important  to  the 
accomplishment  of  their  business  objectives,  (reference  page  36) 

■  Seventy-six  percent  of  respondents  reported  that  DTIC  has  a  “High”  to  "Medium"  impact  in 
supporting  their  overall  mission,  (reference  page  36) 

■  Eighty-nine  percent  of  respondents  would  recommend  DTIC  to  colleagues,  (reference  36) 


4 


Part  II:  Global  Customer  Service 


Performance  Rating  Review 


Overall  Customer  Satisfaction  Performance  Ratings: 

The  2003  CS  survey  findings  showed  that  77  percent  of  respondents  felt  satisfied  with  the  level 
of  DTIC's  customer  care.  When  this  rating  is  compared  to  2002  CS  survey  findings,  the  rating 
declined  4  points.  The  chart  below  illustrates  the  percentage  of  customer  service  performance 
over  a  three  (3)  year  period: 


Comparative  Analysis  of  Customer 
Satisfaction  (Customer  Care)  Performance  Rating 

100-p- - 

80 - ,  - - - 

60 -  -  - 

40 -  -  - 


20- 

FY  2001  % 

FY  2002  % 

FY  2003  % 

DTIC  Composite  Scores 

76 

81 

77 

Fig  2.1 


The  customer  satisfaction  performance  rating  is  generated  by  asking  our  users  to  rate  their  level 
of  satisfaction  with  DTIC's  customer  care  in  eight  (8)  distinct  quality  areas.  Those  critical  care 
elements  and  response  data  are  displayed  below  in  Fig  2.2.  With  regard  to  aspects  of  this  year's 
customer  satisfaction  score  (77  percent ),  all  eight  factors  declined  in  their  rating.  Although  the 
scores  for  Accuracy,  Courtesy,  Helpfulness,  Knowledge  and  Professionalism  remained  fairly 
steady  from  previous  survey  years,  three  (3)  quality  factors  continue  to  be  the  focus  of  interest. 
The  three  quality  factors:  Accessibility,  Responsiveness,  and  Speed  of  Service,  were  the  lowest 
rated  quality  factors  by  the  2003  CS  survey  respondents.  The  ratings  from  these  quality  factors 
appear  to  be  responsible  for  the  marginally  lower  customer  satisfaction  performance  rating  for 
2003.  Fluctuations  in  these  same  three  factors  from  2001  through  2003  appear  to  drive  the 
changes  in  DTIC's  Customer  Satisfaction  Performance  Rating,  both  positively  and  negatively. 
See  Fig  2.2  for  reference. 


5 


DTIC's  Overall  Customer  Care 
Quality  Factors/Elements 

FY  2001 

CSS  Survey 

FY  2002 

CSS  Survey 

FY  2003 

CSS  Survey 

Performance 

Gap 

FY  2002-FY2003 

Accessibility  (Easy  to  access  staff) 

72% 

77% 

72% 

-5 

Accuracy  (Provided  correct  info) 

76% 

80% 

76% 

-4 

Courtesy 

81% 

85% 

82% 

-3 

Helpfulness 

79% 

83% 

79% 

-4 

Knowledge 

77% 

83% 

78% 

-5 

Professionalism 

81% 

85% 

81% 

-4 

Responsiveness(Timely  response) 

72% 

78% 

75% 

-3 

Speed  of  Service 

71% 

77% 

73% 

-4 

Overall  Rating 

76% 

81% 

77% 

Fig  2.  2 


Comparative  Benchmarking  -  Comparison  to  American  Customer  Satisfaction  Index 

(AC  SI) 


DTIC's  Customer  Service  Results: 

■  DTIC’s  customer  service  is  compared  to  the  2002  Federal  Government  American 
Customer  Satisfaction  Index  (ACSI),  which  surveys  53  different  customer  groups  from  39 
different  Federal  government  agencies.  The  ACSI  survey  was  done  in  November  2002,  the 
DTIC  CS  survey  was  done  in  July  2003. 

■  The  2002  December  aggregate  ACSI  score  for  the  Federal  Government  fell  to  70.2  percent 
from  a  record  high  71.3  percent. 

■  DTIC's  composite  data  was  matched  for  comparability  with  the  common  ASCI  index  scoring 
factors. 

■  DTIC’s  2003  Overall  Customer  Satisfaction  Survey  composite  score  is  77  percent. 

■  When  measured  in  terms  of  the  2002  ACSI,  the  DTIC  rating  is  above  the  established  federal 
baseline/benchmark  score  by  +6.8  points. 


Customer  Service 
Elements 

FY  2001 

CSS  Survey 

FY  2002 

CSS  Survey 

FY  2003 

CSS  Survey 

DTIC  Composite  Scores 

76% 

81% 

77% 

ACSI  Federal  Government 
Benchmark* 

68.6% 

71.1% 

70.2% 

DTIC  Excels  by 

+7.4 

+9.9 

+6.8 

*ACSI  is  the  official  service  quality  benchmark  for  the  Federal  Government.  The  cited  survey  was  conducted 
in  November  2002 

Fig  2.3 
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Importance  of  DTIC  to  Accomplishment  of  Business  Objectives  and  Mission  Support: 


Eighty-five  percent  of  all  survey  respondents  indicated  that  DTIC  is  important  to  the 
accomplishment  of  their  business  objectives.  See  Fig  2.4  for  more  details. 
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Fig  2.4 


Seventy-six  percent  of  CS  survey  respondents  reported  that  DTIC  has  a  “high”  to  "medium" 
impact  in  supporting  their  overall  mission  ,  as  compared  to  eighty-two  percent  in  2002.  See  Fig 
2.5  for  more  details. 


O) 

ns 


o 


Q. 


100 

90 

80 

70 

60 

50 

40 

30 

20 

10 


FY  2003  Impact  of  DTIC  in 
Supporting  Users’  Overall  Mission 
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Fig  2.5 
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Recommending  DTIC  Product(s)  and/or  Service(s)  to  Colleagues: 


The  vast  majority  of  CSS  respondents  (89  percent)  would  recommend  DTIC  to  colleagues.  This 
finding  is  basically  consistent  with  previous  survey  years. 


Comparative  Analysis  on 
Recommending  DTIC  to  Colleagues 
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Fig  2.6 
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Part  III:  DTIC’s  Offerings 
DTIC  Online  Services 
DTIC  Online  Services  Overall  Quality  and  Performance: 

The  2003  CS  survey  respondents  were  asked  to  rate  their  level  of  satisfaction  with  DTIC  online 
services.  About  two-thirds  rated  the  quality  as  "Very  Good"  to  "Excellent."  The  overall 
percentage  of  users  who  were  satisfied  with  DTIC  online  services  increased  1 1  percentage  points 
from  58  percent  in  2002  to  69  percent  in  2003.  Note:  Overall  favorable  percentages  are  based 
on  normalized  data.  These  may  vary  somewhat  from  the  total  percentage  of  favorable  responses. 


Comparative  Analysis  of  DTIC’s  Overall  Online  Services  Performance  Rating 


CS  Survey 

Overall 

Favorable 

Neutral 

Unfavorable 

2003 

69% 

65% 

28% 

7% 

2002 

58% 

54% 

21% 

25% 

2001 

68% 

60% 

32% 

8% 

Fig  3.1 


The  top  job  positions  in  which  the  2003  CS  survey  respondents  most  frequently  categorized 
themselves  were  (1)  Librarians,  (2)  Engineers,  (3)  Researcher/ Analyst/Program  Managers  (see 
Part  IV  -  Demographics  for  details).  All  three  (3)  top  job  positions  showed  similar  patterns  of 
responses  in  comparison  to  the  overall  CS  online  perfonnance.  The  2003  online  services 
performance  data  reflects  a  notable  favorable  rating  increase,  in  comparison  to  2002.  It  should 
be  pointed  out  that  the  count  of  respondents  in  2002  for  these  three  top  job  positions  was  small, 
resulting  in  large  percentage  shifts.  The  following  identifies  the  online  services  quality  rating 
segmented  by  the  top  three  user  groups  in  more  detail. 


Analysis  of  DTIC’s  Online  Services  Performance  Rating  by  User  Group 


FY  2003  CS  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

65% 

28% 

7% 

Librarian 

65% 

28% 

7% 

Engineers 

62% 

29% 

9% 

Researcher/Analyst 

67% 

24% 

10% 

FY  2002  CS  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

54% 

21% 

25% 

Librarian 

63% 

26% 

11% 

Engineers 

39% 

17% 

44% 

Researcher/Analyst 

53% 

17% 

30% 

Fig  3.2 


DTIC  Online  Service  Usage: 

Private  STINET  was  created  by  the  merger  of  Secure  STINET  and  WED,  and  implemented 
three  (3)  months  before  the  2003  CS  survey.  Based  on  write-in  comments  from  the  2003  CS 
survey,  usage  data  reflect  that  most  online  customers  reported  that  they  currently  use  Private 
STINET. 
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DTIC  Homepage  Review 


DTIC  Homepage  Usage 

The  majority  of  CS  survey  respondents  (75  percent)  indicated  they  have  accessed  the  DTIC 
Homepage  in  the  past  12  months.  This  finding  continues  to  reflect  no  significant  change  for 
DTIC  Homepage  usage  from  2001.  The  level  of  access  among  the  top  (three)  3  job  positions 
(detailed  in  Part  IV  Demographics)  shows  80  percent  (each)  of  the  Librarians  and  Engineers 
utilizing  the  DTIC  Homepage  in  the  past  12  months.  Following  closely  were  the 
Researcher/ Analyst  at  72  percent.  It  should  be  noted  that  the  DTIC  Homepage  was  redesigned 
after  the  2002  Survey  and  is  strictly  intended  for  the  general  public.  Content  for  registered  users 
is  planned  for  later  in  2003. 


Fig  3.3 

Note:  Comparable  data  not  available  for  2002 


DTIC  Homepage  Quality  and  Performance 

The  2003  CS  survey  respondents  were  asked  to  rate  their  satisfaction  with  respect  to  5  quality 
factors  in  relation  to  the  DTIC  Homepage.  For  2003,  the  majority  of  the  CS  survey  respondents 
rated  the  DTIC  Homepage  favorably,  with  an  overall  satisfaction  rating  of  69  percent.  The  top 
rated  quality  factors  for  the  Web  site  were:  "Accessibility"  at  74  percent  and  following  at  a 
close  second  was  "Appearance"  at  72  percent.  When  comparing  results  from  2001,  overall,  there 
were  no  profound  changes  in  respondents'  evaluation  of  the  DTIC  Homepage  as  a  whole. 
However,  it  may  be  a  hopeful  sign  that  negative  views  of  the  "content"  quality  factors  dropped 
nine  points  between  2001  and  2003.  See  Fig  3.4  for  more  details. 
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Comparative  Analysis  of  DTIC  Homepage  Performance  Rating 


FY  2003  Homepage  Quality  Factors 

Overall 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Easy  to  access  site) 

74% 

74% 

22% 

5% 

Ease  of  Use  (Navigating  the  site) 

65% 

59% 

26% 

16% 

Organization 

67% 

60% 

26% 

14% 

Content 

70% 

68% 

25% 

8% 

Appearance  (How  doe  the  site  look) 

72% 

68% 

27% 

5% 

Overall  Averages 

69% 

66% 

25% 

9% 

FY  2001  Homepage  Quality  Factors 

Overall 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Access  to  Site) 

77% 

81% 

16% 

3% 

Ease  of  Use  (Navigating  the  Site) 

65% 

56% 

30% 

15% 

Organization  (Logically  arranged) 

63% 

51% 

31% 

18% 

Content  (Helpful,  focused, 
understandable) 

65% 

56% 

27% 

17% 

Usability  (Information/service  useful) 

67% 

59% 

32% 

9% 

Overall  Averages 

67% 

61% 

27% 

12% 

Note:  Quality  factors  differ  slightly  for  complete  comparison  for  2001/2003.  Comparable  data  are 
not  available  for  2002. 


Fig  3.4 


With  regard  to  aspects  of  the  satisfaction  levels  of  DTIC's  Homepage  from  the  2003  CS  survey 
according  to  the  top  3  job  positions,  crosstab  data  (not  presented  in  this  report)  shows  Librarians 
were  noticeably  more  pleased  with  our  homepage  appearance,  ease  of  use,  organization  and 
content  than  were  other  respondents.  For  more  details,  the  following  identifies  the  5  quality 
ratings  of  the  DTIC  Homepage,  segmented  by  the  top  3  user  groups,  consisting  of  Librarians, 
Engineers,  and  Researcher/ Analysts. 
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"Accessibility"  Overall  Ratings: 


•  73.5  percent  of  Users  rated  accessibility  as  “Very  Good”  to  “Excellent”  (Favorable) 

•  21.8  percent  as  “Good”  (Neutral) 

•  4.8  percent  as  “Fair”  to  “Poor”  (Unfavorable) 


Analysis  of  DTIC  Homepage  Performance  Rating  -  Accessibility  (Easy  to  Access  Site) 


FY  2003  CS  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

74% 

22% 

5% 

Librarian 

73% 

20% 

7% 

Engineers 

74% 

21% 

5% 

Researcher/Analyst 

71% 

26% 

2% 

Fig  3.5 


"Appearance"  Overall  Rating: 

•  68  percent  of  users  rated  appearance  as  “Very  Good”  to  “Excellent”  (Favorable) 

•  27  percent  as  “Good”  (Neutral) 

•  5  percent  as  “Fair”  (Unfavorable) 


Analysis  of  DTIC  Homepage  Performance  Rating  -  Appearance  (How  Does  the  Site  Look) 


FY  2003  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

68% 

27% 

5% 

Librarian 

78% 

19% 

4% 

Engineers 

61% 

33% 

6% 

Researcher/Analyst 

63% 

33% 

4% 

Fig  3.6 


"Ease  of  Use"  (Navigating  the  site)  Overall  Ratings: 

•  58.8  percent  of  users  rated  navigation  as  “Very  Good”  to  “Excellent”  (Favorable) 

•  25.7  percent  as  “Good”  (Neutral) 

•  15.5  percent  as  “Fair”  (Unfavorable) 


Analysis  of  DTIC  Homepage  Performance  Rating  -  Ease  of  Use  (Navigating  the  Site) 


FY  2003  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

59% 

26% 

16% 

Librarian 

68% 

18% 

14% 

Engineers 

57% 

27% 

16% 

Researcher/Analyst 

53% 

33% 

14% 

Fig  3.7 


12 


"Organization"  Overall  Rating: 

•  60.3  percent  of  users  rated  organization  as  “Very  Good”  to  “Excellent”  (Favorable) 

•  26.1  percent  as  “Good”  (Neutral) 

•  13.7  percent  as  "Fair"  (Unfavorable) 


Analysis  of  DTIC  Homepage  Performance  Rating  -  Organization 


FY  2003  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

60% 

26% 

14% 

Librarian 

68% 

20% 

12% 

Engineers 

59% 

27% 

14% 

Researcher/Analyst 

54% 

31% 

15% 

Fig  3.8 


"Content"  Overall  Rating: 

•  67.6  percent  of  users  rated  content  as  “Very  Good”  to  “Excellent”  (Favorable) 

•  24.7  percent  as  “Good”  (Neutral) 

•  7.7  percent  as  “Fair”  (Unfavorable) 


Analysis  of  DTIC  Homepage  Performance  Rating  -  Content 


FY  2003  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

68% 

25% 

8% 

Librarian 

75% 

20% 

4% 

Engineers 

66% 

28% 

6% 

Researcher/Analyst 

60% 

33% 

7% 

Fig  3.9 
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Performance  of  DTIC  Homepage  by  Top  3  User  Groups: 


Figures  3.10  to  3.12  depict  the  overall  rating  of  various  quality  factors  for  the  DTIC  Homepage 
for  the  2003  CS  survey. 


FY  2003  DTIC  Homepage  Quality  Performance  Rating 

Librarian 

Accessibility 

Appearance 

Ease  of  Use 

Organization 

Content 

Overall  Average 

- 1  7^ 

0 

- 1  74 

1 

- 1  R7 

- 1  68 

_ 1  1  z 

_ i  /i 

10  20  30  40  50  60  70  80  90  1C 

(%) 

Fig  3.10 


FY  2003  DTIC  Homepage  Quality  Performance  Rating 

Engineers 


(%) 


Fig  3.11 

FY  2003  DTIC  Homepage  Quality  Performance  Rating 
Researcher/Analyst 


Fig  3.12 
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DTIC  Annual  Users  Conference 


The  majority  of  2003  CS  survey  respondents  reported  that  they  have  not  attended  an  annual 
DTIC  Users  Conference  in  the  past  three  (3)  years.  Among  the  12  percent  of  CS  survey 
respondents  who  attended  the  conferences,  generally  they  found  the  event  valuable  in  terms  of 
the  information  provided.  According  to  write-in  comments,  a  number  of  users  reported  they 
preferred  Spring  (April)  Annual  Users  Conferences  in  comparison  to  Fall  (November) 
conferences.  Comparable  data  are  not  available  for  previous  years.  2003  results  are  shown  in 
Figure  3.13. 


Data  not  gathered  for  previous  years 


Fig  3.13 


A  Sample  of  2003  CS  Survey  Respondents’  Comments  on  the  DTIC  Annual  Users 

Conference 

Significant  user  comments  regarding  the  annual  conference  are  provided  below: 

■  "The  conference  always  provides  current  and  useful  information  and  is  invaluable  in  helping 
me  to  perfonn  my  job  successfully." 

■  "I  much  prefer  the  April/Spring  date  to  the  fonner  Fall  time  -  makes  travel  much 
better/easier!  And  a  nicer  time  to  visit  D.C.  Session  content  remains  good/pertinent,  and 
tours  are  excellent.  Luncheon  speakers  still  interesting/entertaining.  Exhibits  seem  to  have 
declined  in  quality. . . " 

■  "Why  would  I  attend  this  conference  in  addition  to  SLA,  MLW,  and  other  competing 
conferences?" 


"The  conference  was  very  helpful  -  it  was  beneficial  to  be  able  to  give  DTIC  suggestions  and 
also  see  the  current  development  of  Private  STINET." 
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■  "Regretfully,  no  funds  here  to  attend  anything. . . " 

■  "I  would  like  to  see  more  on-line  training  held  during  the  DTIC  Annual  Users  Conferences." 
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Part  IV:  User  Demographics 


Job  Position  Status  and  Organization  Type: 

The  customer  profile  in  this  survey  represents  five  (5)  organizational  groups  and  seven  (7)  job 
positions,  the  distribution  of  which  is  shown  in  Figures  4. 1  and  4.2.  Aspects  of  the  CS  survey 
findings  during  2001  through  2003  continue  to  show  similar  organizational  status  percentages, 
indicating  that  the  majority  of  respondents  were  affiliated  with  the  Department  of  Defense  (DoD) 
(53  percent).  Following  at  33  percent  were  the  Industry  organizations  (Government  Contractors, 
Corporate,  etc.).  Overall,  the  respondents'  top  four  (4)  organizational  types  remain  consistent 
with  previous  survey  years. 


User  Organization  Type 


Organization 

FY  2001  CS 
Survey 

FY  2002  CS 
Survey 

FY  2003  CS 
Survey 

DoD-Military/Civilian 
Services  Agencies 

52% 

48% 

53% 

Industry  / Gov't.  Contractor 

35% 

38% 

33% 

Non-DoD  Federal  Gov't. 

1% 

5% 

4% 

Academia:  College/Univ. 

11% 

3% 

5% 

HBCU/MI 

N/A* 

3% 

2% 

Other 

N/A* 

N/A* 

3% 

*Data  not  gathered  in  previous  survey 

Fig  4.1 


The  top  three  (3)  job  positions  showed  a  similar  pattern  of  responses  throughout  the  surveyed 
years.  This  year,  the  largest  number  of  respondents  classified  their  job  position  as  Librarian  (24 
percent).  Following  at  17  percent  were  the  Engineers,  and  Researcher/ Analysts  at  16  percent. 
Overall,  these  findings  continue  to  reflect  no  significant  changes  to  job  position  from  previous 
surveyed  years.  Crosstab  data  (not  presented  in  this  report)  reflects  the  largest  number  of 
Librarians  and  Technical  Information  Specialists  described  their  current  organization  as  DoD. 
However,  it  is  interesting  to  note  that  the  majority  of  this  year’s  Researcher/ Analyst  respondents 
describe  their  current  organization  as  coming  from  DoD  agencies,  whereas  previous  surveyed 
years  reflected  Industry  organizations. 
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Job  Position 


Job  Position 

FY  2001  CS 
Survey 

FY  2002  CS 
Survey 

FY  2003  CS 
Survey 

Librarian 

16% 

25% 

24% 

Researcher/ Analysts/Program 
Manager 

18% 

16% 

16% 

Engineer 

16% 

17% 

17% 

Scientist 

14% 

9% 

11% 

Technical  Info  Specialist 

8% 

9% 

10% 

Contracting  Officer 

2% 

!  5% 

3% 

Resource 

Management/Financial/ 

Budget  Analyst 

4% 

4% 

3% 

Security 

8% 

N/A* 

N/A* 

Other 

14% 

15% 

17% 

Note:  Greyed  entries  reflect  the  top  three  positions  in  terms  of  the  number  of  survey  respondents. 


*Data  not  collected  Fig  4.2 


Intermediary  vs.  End  Users: 

The  two  (2)  categories  listed  in  the  chart  below  reflect  areas  on  how  our  respondents  acquire 
scientific  and  technical  information.  The  2003  CS  survey  shows  a  continuing  trend  of 
respondents  primarily  acting  as  infonnation  providers  to  others  as  illustrated  in  Fig  4.3.  It  should 
also  be  noted  that  this  year,  crosstab  data  shows  48  percent  of  the  DoD  (Military/Civilian) 
agencies  reported  themselves  as  end  users  and  5 1  percent  as  intermediary  users  — almost  evenly 
split.  Last  year’s  finding  overwhelmingly  had  the  DoD  agencies  reporting  themselves  as  end 
users  at  70  percent. 


100-f 

80- 

S>  60- 

(0 

8  40- 

0) 

CL 

20- 

0-^ 


■  Intermediary  Users 

□  End  Users 

□  Other 


Note:  2001  responses  to  this  question  differ  slightly:  four  choices  were  available  to  respondents  including 
a  selection  for  "other."  To  incorporate  comparable  data  for  2002  reporting,  we  combined  each  of  2001 
responses  of — "both  end  user  and  information  provider"  and  "intermediary"  for  one  response-intermediary 
Users. 

*Data  not  gathered  in  previous  survey  Fig  4.3 


FY  2001  % 


Comparative  Analysis  on 
Intermediary  vs.  End  User 


3a 


64 


32 


FY  2002  % 


65 


35 


N/A* 


FY  2003  % 


55 


45 


N/A* 
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Time  as  a  Registered  User: 

The  results  from  this  year's  survey  show  that  the  largest  number  of  respondents  have  been 
registered  users  for  "1-2  years."  Although  the  largest  number  of  respondents  came  from  this 
category,  the  overall  picture  shows  that  respondents  have  been  registered  with  DTIC  for  the  full 
range  of  time  increments.  Fifty-one  (5 1)  percent  of  users  responding  to  the  survey  reported 
having  been  registered  with  DTIC  two  (2)  years  or  less,  while  49  percent  reported  three  (3)  years 
or  more.  Also  to  be  noted  are  the  results  of  the  crosstab  data  of  the  2003  Librarian  respondents 
representing  the  largest  number  of  “6  or  more  years”  as  registered  users.  The  Engineers 
represented  the  majority  of  the  “1  to  2  years"  as  registered  users.  The  data  displayed  in  Fig  4.4 
below  defines  respondent  time  as  a  registered  DTIC  user. 
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Comparative  Analysis  on 
Time  Data 


- 

H 

u- 

FY  2002  % 

FY  2003  % 

□  Less  than  1  year 

17 

10 

□  1-2  years 

29 

41 

□  3-5  years 

21 

24 

□  6  or  more  years 

33 

25 

Fig  4.4 

Note-  different  measurement  scales  (time  increments)  were  used  for  2001 
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Part  V :  Communications,  Access  and  Information  Requirements 


Information  Needs: 

The  vast  majority  of  users  (77  percent)  indicated  a  high  level  of  satisfaction  with  their 
information  needs  being  met  by  searching  DTIC's  collection.  The  2003  CS  survey  respondents 
were  also  asked  to  provide  specific  comments  regarding  their  current  and  future  informational 
needs.  The  information  needs/topic  areas  respondents  most  often  mentioned  continue  to  be 
access  to  full  text  electronic  documents,  as  was  also  reflected  in  last  year's  findings.  It  is  also 
interesting  to  note  that  results  of  the  2002  CS  survey  open-ended  comments  included  a  number 
of  information  needs  centered  on  "Homeland  Defense/Security;"  however,  2003  CS  respondents 
hardly  mentioned  the  need  for  this  type  of  information.  Based  on  the  frequency  with  which 
informational  needs  were  addressed,  outlined  below  are  the  top  3  informational  needs  expressed 
by  respondents  in  their  written  comments: 

■  Full  Text  Access 

■  Historical  Data  (Anny  Topics,  Chemical  Toxicity) 

■  Medical  (Military  Medical,  Biomedical) 

Communication  Preferences  and  Satisfaction: 


For  two  (2)  years  in  a  row  the  CS  surveys  find  respondents  indicating  the  Internet/Online  as  the 
preferred  communication  method.  There  appears  to  be  a  trend  away  from  the  telephone  for  CS 
respondents  overall.  It  is  also  interesting  to  note  that  crosstab  data  shows  Librarian  respondents 
preferred  the  telephone  (47  percent)  but  not  overwhelmingly.  It  should  be  pointed  out  that  when 
you  combine  the  numbers  for  the  electronic  communication  (e-mail  and  Internet),  5 1  percent  of 
Librarians  preferred  this  type  of  medium  when  accessing  DTIC. 


100 

90 

80 

70 

60 

50 

40 

30 

20 

10 


Comparative  Analysis  of  Access 
Medium  (User  Contact  Preference) 


HE 


l 


0- 

FY  2001  % 

FY  2002  % 

FY  2003% 

■  Telephone  (Commercial/DSN) 

36 

33 

28 

□  Internet/Online 

25 

39 

44 

■  E-mail 

35 

25 

24 

□  Fax 

3 

1 

1 

H  Mail 

N/A 

1 

1 

□  Other 

2 

1 

2 

Fig  5.1 
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Effectiveness  of  Corporate  Communications: 


When  users  were  asked  how  effective  DTIC  marketing  communication  efforts  are,  the  majority 
of  respondents  (74  percent)  “agree”  to  “strongly  agree”  that  DTIC  does  an  adequate  job  in 
communicating  (transmitting/mailing)  information  about  new,  changed  and/or  enhanced 
products  and  services  offerings,  etc.  The  2003  CS  survey  results  on  the  effectiveness  of 
corporate  communication  show  a  similar  pattern  of  responses  in  comparison  to  last  year's 
findings.  Communication  about  submitting  documents  is  still  perceived  as  relatively  weak, 
though  improving. 

Again,  note  that  overall  favorable  percentages  are  based  on  normalized  data,  a  process  which 
uses  the  total  number  of  valid  responses  along  a  five  point  scale  and  converts  these  to  a  100  point 
scale.  These  may  vary  somewhat  from  the  total  percentage  of  favorable  responses.  The  overall 
favorable  percentages  are  derived  from  the  “Mean  (normalized)”  column  in  the  tables  for 
quantitative  results  at  Appendix  B. 


Comparative  Analysis  on  Effectiveness  of  Corporate  Communication 


FY  2001  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product/Service  Changes 

* 

* 

* 

Product/Service  Changes 

81% 

4% 

15% 

Training  Opportunities 

63% 

5% 

32% 

Submitting  Documents 

46% 

9% 

45% 

Ordering  Documents 

* 

* 

* 

System  Disruptions/Downtime 

* 

* 

* 

Overall  Rating 

61% 

13% 

27% 

FY  2002  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product/Service  Changes 

83% 

0.7% 

16% 

Product/Service  Changes 

82% 

2% 

16% 

Submitting  Documents 

59% 

4% 

37% 

Training  Opportunities 

71% 

3% 

27% 

Ordering  Documents 

77% 

3% 

20% 

System  Disruptions/Downtime 

76% 

3% 

22% 

Overall  Rating 

75% 

3% 

23% 

FY  2003  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product/Service  Changes 

81% 

5% 

14% 

Product  /Service  Changes 

82% 

6% 

12% 

Submitting  Documents 

63% 

8% 

29% 

Training  Opportunities 

67% 

11% 

22% 

Ordering  Documents 

75% 

8% 

17% 

System  Disruptions/Downtime 

76% 

7% 

17% 

Overall  Rating 

74% 

8% 

18% 

Fig  5.2 

Note:  Inquiries  on  the  specifics  above  varied  year  to  year. 
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Preferred  Method  of  Ordering  Documents: 

The  2003  CS  survey  asked  for  the  preferred  method  of  ordering  documents.  Over  half  of  the 
respondents  (59  percent)  reported  that  Intemet/Online  Services  was  the  preferred  medium  of 
ordering  documents.  Following  with  13  percent  was  e-mail.  These  numbers  show  similar 
patterns  of  responses  in  comparison  to  last  year's  findings.  See  Fig.  5.3  for  more  details. 
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Users’  Preferred  Method  of  Ordering 

Documents 

■  Commercial  Telephone/DSN 

□  E-mail 

□  Fax 

□  Internet/Online  Services 

□  Mail 

■  Other 


FY  2002  % 

13 

14 
4 
52 
13 
4 


FY  2003  % 
9 
13 
2 

59 

13 

4 


Data  not  gathered  in  2001 


Fig  5.3 
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Part  VI:  Selected  User  Comments 


The  following  statements  were  selected  based  on  the  frequency  with  which  an  issue  was 
addressed,  either  positively  or  negatively,  and  the  importance  of  the  issue  as  determined  by  the 
survey  analysts.  Comments  about  individual  staff  are  generally  omitted. 

The  comments  below  represent  a  selected  sample  of  quotes  from  users. 


jFavorable  User  Comments: 


■  “Excellent  service.  Stays  abreast  of  the  rapidly  changing  IT  technologies.  Always 
attempting  to  make  it  more  customer  friendly  and  improve  satisfaction.” 

■  “Whenever  I  have  problem  (I  often  do— due  to  my  accessing  the  system  infrequently)  I  call 
customer  service  and  I  get  excellent  helpers  trying  to  resolve  the  issue  the  same  day  and  I 
like  that  very  much.  Because  I  call  them  when  I  have  to  help  a  customer  and  find  then  only 
have  the  problem  accessing  the  system  and  my  customer  is  waiting.  Without  getting 
embarrassed,  I  get  help  and  I  can  help  my  customer  in  turn  quickly  saves  me  face.  Keep  it 
up.” 

■  “The  good  service  is  really  appreciated.  It  helps  make  our  day  smoother  when  the  people 
who  help  us  treat  us  with  courtesy  and  kindness.  Thanks  to  all  those  who  go  above  and 
beyond  with  customer  service  skills.” 

■  “I  have  interacted  with  DTIC  many  times  from  registration  of  contracts  to  ordering  of 
documents.  I  am  very  impressed  with  their  quick  responsiveness  when  returning  my  calls 
and  assisting  me  with  my  questions.  When  I  order  a  document,  I  am  impressed  at  the  quick 
time  that  I  received  the  document  after  ordering  (often  within  a  week).” 

■  “My  phone  calls  to  DTIC  are  returned  within  a  day  (at  the  latest).  The  personnel  are 
professional  and  helpful,  and  solve  problems  in  a  timely  manner.” 

■  “In  addition,  DTIC  personnel  diligently  check  the  quality  of  the  papers  and  reports  submitted 
to  them  by  our  organization.  We  receive  calls/e-mails  requesting  better  copies  or  missing 
pages.  This  step  ensures  the  best  quality  product  is  available  for  our  customers.” 

■  “Was  very  impressed  when  I  encountered  a  problem  that  I  received  an  expeditious  response 
from  several  Associates  who  were  very  willing  to  assist  me.  I  also  received  follow-up  calls 
to  ensure  that  I  was  able  to  obtain  the  infonnation  I  needed.  Very  commendable!  You  have 
excellent  Associates  willing  to  go  the  extra  mile.  Thanks!” 

■  “They  are  the  best  group  of  people  I've  done  business  with  in  all  the  years  I’ve  worked  (1989) 
for  the  military!” 

■  “The  new  web-site  is  wonderful  and  the  customer  service  representatives  are  most 
understanding,  helpful  and  courteous.” 
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■  “Customer  care  is  excellent  from  DTIC.  I  have  always  received  the  infonnation  requested  in 
a  timely  manner.  When  the  information  is  not  available  I  received  a  reply  electronically  or 
from  a  live  person.  On  a  scale  of  1  to  10  ten  being  the  best  I  rate  the  customer  care  services 
with  10.” 

■  “10. 1  have  always  been  impressed  with  the  DTIC  organization.  I  have  used  it  for  a  number 
of  years,  and  have  found  that  DTIC  provides  a  large  number  of  services  and  materials  for  a 
very  reasonable  amount  of  money.  I  try  to  make  sure  our  staff  and  students  are  aware  of  the 
organization.  It  is  an  essential  organization  to  the  DoD  community.” 

Analysis:  The  Positives 

Many  customers  are  pleased  with  DTIC's  overall  customer  service.  Quick  response  time  to 
inquiries  were  mentioned  as  well  as  many  references  regarding  the  polite  and  courteous  staff. 
Some  of  DTIC's  customers  even  stated  that  DTIC  employees  go  the  "extra  mile"  when  providing 
customer  service.  Several  DTIC  employees  were  singled  out  in  particular  for  the  high  level  of 
customer  care  provided. 

junfavorable  User  Comments:  j 


■  “DTIC  sends  out  frequent  e-mail  messages  to  notify  online  users  of  an  interruption  in 

service,  etc.  This  message  can  be  annoying  to  those  who  not  rely  on  DTIC  for  constant 
access.  Also,  I  have  commonly  received  duplicate  messages  from  at  least  two  parties  in  the 
past  (perhaps  someone  from  DTIC,  and  someone  from  STINET?).  It  would  be  best  to 
consolidate  such  outgoing  e-mail  messages  to  one  person  in  order  to  reduce  this  duplication, 
if  this  has  not  already  been  done.” 

■  “I  have  experienced  issues  with  double  or  incorrect  billing  for  documents.  Each  time 
customer  service  reps  were  unable  to  issue  a  refund  without  supervisory  intervention. 
Additionally,  I  have  had  to  use  a  supervisor  to  obtain  a  receipt  for  one  set  of  documents.” 

■  “Although  the  CS  reps  are  very  courteous,  there  seems  to  be  a  significant  disconnection 
between  invoicing  and  billing.  There  seems  to  be  one  person  for  each  job  and  no  one  appears 
to  be  cross  trained.  As  a  result,  when  a  person  is  out  of  the  office,  there  is  no  one  else 
available  to  help  resolve  issues  in  a  timely  manner”. 

■  “Phone  calls  are  hard  to  get  through  to  a  person.  East  coast  times  do  not  correlate  with  West 
coast  work  days.  Rarely  receive  a  call  back  when  a  problem  has  arisen  with  tech  control, 
even  though  they  say  "we  will  get  back  to  you." 

■  “Customer  service  is  inconsistent.  People  do  not  always  call  back,  so  I  have  to  continue  to 
call  DTIC.  Even  if  someone  says  the  deposit  account  will  be  credited,  it  often  is  not  done  the 
first  time.  Further  phone  calls  ensue.  Form  55s— I  end  up  finding  out  who  the  releasing 
agency  POC  is,  so  I  can  follow  up  myself;  I  thought  that  was  DTIC's  job.  If  there  is  a 
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problem  with  an  online  order,  why  does  DTIC  send  a  postcard?  E-mail/fax/phone  should  be 
used  before  USPS,  which  is  the  slowest  mode  of  communication.” 

■  “The  transition  from  Web  Enabled  DROLS  to  Private  STINET  was  very  confusing.  I  didn’t 
remember  that  I  already  signed  up  months  earlier  and  I  only  needed  to  reset  my  password.  I 
just  noticed  that  I  couldn't  log  in,  and  I  couldn’t  find  the  Private  STINET  home  page  from  the 
DTIC  home  page  (all  links  seemed  to  lead  to  Public  STINET).  Therefore,  I  submitted  a 
signup  request,  which  took  a  lot  of  time  and  delay  me  at  a  point  when  I  really  needed  DTIC. 

I  made  some  errors  in  my  signup  request,  so  I  resubmitted  it  and  call  Customer  Service  to  tell 
them  to  use  the  second  one;  they  didn’t  notice  that  I  was  sign  up  for  Private  STINET.  Finally 
when  both  signup  requests  were  denied  I  called  and  found  out  why:  I  was  already  signed  up 
and  only  needed  to  reset  my  password.  This  delayed  me  a  few  days  at  a  time  when  I  urgently 
needed  to  use  Private  STINET  for  a  literature  search.” 

■  “Frequently  have  trouble  accessing  the  website  due  to  continually  change  in  security 
standards,  which  require  an  inordinate  amount  of  time  each  time  I  attempt  to  access,  since  I 
don’t  access  every  week.” 

■  “I  remain  disappointed  with  someone  getting  back  to  me  when  I  leave  phone  mail.  Recently 
I  placed  an  overnight  or  rush  order,  and  it  was  3  days  till  someone  got  back,  stating  that  the 
phone  system  had  problems  and  they  had  just  gotten  my  phone  message.  Very  unhappy 
about  that.  Staff  was  apologetic,  but  my  engineers  don’t  like  apologies.  I  also  reported  an 
erroneous  telephone  number  some  months  ago,  and  I  don't  think  your  website  reflects  the 
correct  number. .  .yet.  DTIC  service  has  deteriorated  significantly  over  the  past  5  years.” 

■  “I  find  it  very  difficult  to  understand  what  it  is  that  DTIC  does  and  how  to  go  about  using  its 
services.  I  do  a  lot  of  searching  the  Internet  for  technical  infonnation  and  I  find  the  DTIC 
site  one  of  the  worst  to  use.  I'm  not  exactly  sure  why,  but  I  do  know  that  it  is  not  very  user 
friendly.” 

■  “We  have  tried  to  pay  with  a  credit  card  to  eliminate  this  deposit  account  charge,  but  still 
seem  to  be  charged  with  a  $25  fee  each  month.  DTIC  could  do  a  better  job  of  conveying 
payment  options.  Also,  we  are  VERY  unhappy  about  the  way  DTIC  posts  charges  to  the 
Govt.  Purchase  Card  Accounts.  We  might  for  example  order  30  titles  at  $7  each.  Instead  of 
posting  the  transaction  as  $210  on  our  purchase  card,  we  are  seeing  30  $7  postings.  If  there 
are  multiple  orders,  and  the  titles  come  in  a  few  at  a  time  or  in  multiple  shipments  or 
whatever  combination,  it  makes  it  very  difficult  to  tell  which  part  of  the  orders  the  charges 
are  paying.  There  is  absolutely  nothing  that  will  help  us  to  identify  the  purchase  . . .  nothing. 
Could  you  imagine  going  to  the  grocery  store,  paying  with  a  credit  card  and  then  having  the 
grocery  store  post  the  individual  items  as  separate  transaction  t  your  account  instead  of  the 
whole  purchase?  It  would  be  a  nightmare  for  sure.  This  what  DTIC  is  doing  to  us  and  for  an 
organization  that  says  it  wants  to  get  rid  of  the  Deposit  Accounts,  this  practice  of  DTIC's  is 
making  us  rethink  the  whole  decision  to  stop  using  a  deposit  account.  If  you  would  post  the 
charges  as  a  shipment  is  sent  out  for  all  the  items  in  that  shipment,  it  would  make  the  process 
about  1000%  better.  For  example,  if  you  send  out  a  shipment  which  has  5  $7  items  and  3 
$12  items,  post  the  transaction  as  $71  instead  of  as  7,  7,  7,  7,  7,  12,  12,  12.  It  is  much  easier 
to  identify  a  $7 1  purchase  than  the  above  "don’t  they  all  look  alike,  can't  tell  them  apart" 
postings.  If  you  want  further  documentation  of  this  problem,  please  call  (name  omitted). 
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Because  of  a  vacancy  in  our  office,  we  have  been  unable  to  order  much  of  anything,  but  we 
haven’t  really  felt  too  badly  about  it  because  of  the  billing  thing.  We  now  have  several 
hundred  titles  backlogged  to  order  and  are  reluctant  to  place  the  orders  because  of  the 
horrendous  prospect  of  dealing  with  reconciling  our  credit  card  statements!  Hoping  you  can 
give  us  some  good  new  about  this  issue.” 

■  “I  would  recommend  DTIC  only  because  of  its  unique  content  not  because  it  is  'user 

friendly.'  It  is  an  extremely  difficult  page(s)  to  navigate.  The  usual  searcher/librarian  clues 
of  'search  here'  are  missing.  The  nomenclature  is  awkward  &  different  from  any  other 
database  that  I  search.  Obviously,  I  am  frustrated  with  the  DTIC  'collections'  and  dread 
attempting  to  locate  reports  my  clients  need.  I  do  like  the  full-text  in  PDF  feature.  My 
clients  are  delighted  with  the  immediate  access  that  feature  permits  me  to  provide.” 


Analysis:  What  Needs  Work 

■  Some  customers  are  still  experiencing  problems  understanding  what  DTIC  is  all  about. 

■  Web  site  is  still  hard  to  navigate  and  not  user-friendly,  according  to  some  of  our  customers. 

■  DTIC's  communication  processes  remain  a  concern  with  DTIC's  customers: 

-  DTIC's  phone  system  is  an  irritation  to  our  customers  because  it  is  difficult  to  speak  to  a 
"live"  person,  especially  when  there  is  a  time  zone  difference. 

-Leaving  voice  mail  messages  and  receiving  no  response  is  still  an  issue  with  some. 

■  DTIC  employees  should  be  cross-trained,  so  more  than  one  individual  can  respond  to  various 
types  of  questions. 

■  Billing  problems  still  constitute  a  major  concern  with  some  of  our  customers  as  well. 

■  Transition  to  Private  STINET  was  not  as  smooth  as  some  of  our  customers  had  anticipated. 
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Part  VII:  Issues  for  Further  Study,  Conclusions  and  Recommendations 


1 .  The  following  suggestions  were  made  by  users  responding  to  the  survey: 


"Try  to  have  some  coverage  of  phones  for  new  problems  through  the  business  day.  (I  have 
found  some  difficulty  reaching  a  live  person  to  discuss  new  problems  late  in  the  day.)  " 

"Update  your  automatically  mailed  notices  to  more  accurately  convey  their  information.  One 
recent  example:  putting  a  new  contract  on  the  DTIC  account  resulted  in  a  confusing  notice 
saying  that  contract  had  "been  extended."  The  staff  helped  explain  what  this  meant,  but  with  a 
little  programming  effort,  such  confusion  could  be  avoided  in  the  first  place.” 

“Private  STINET  (Secure  STINET)  good  improvement.  Need  better,  easier  to  use,  Google-like 
search  capability.  Would  like  to  have  text  books,  reference  books,  and  similar  on-line 
information  capabilities  (eg.  Schaum's  outline  perhaps?)” 

“Your  quarterly  bulletins  (on  paper)  were  very  infonnative  and  helpful.  Any  chance  of 
reinstituting  them?  Even  if  you  publish  them  electronically  and  inform  DTIC  users  via  listservs 
or  general  e-mail  of  the  link(s)  to  access  them,  it  would  be  very  helpful.  I  think  you  have  more 
services  than  the  average  user  is  aware  of.  With  budgets  being  cut  all  the  time  and  users  being 
unable  to  attend  DTIC's  annual  conference  every  year,  an  online  version  of  Dr.  Wright's  updates 
would  be  very  helpful,  plus  periodic  reminders  of  what  services  you  do  provide.” 

“Eliminate  requirement  to  enter  password  a  second  time  to  view  a  citation.  Initial  login  should 
be  sufficient." 

"  Should  have  same  index  coverage  as  CD-ROM  or  provide  online  option  to  input  requests  for 
older  documents.  Currently  a  two  step  process  for  us;  order  what  we  can  online,  then  call  in 
order  for  DTIC  docs  not  on  system." 

"  Convert  older,  requested  docs  for  PDF  to  add  to  online  system.  For  example,  after  X  number 
of  requests  with  Y  timeframe  a  document  is  automatically  queued  for  PDF  conversion, 
regardless  of  document  age." 

"ECABs:  Allow  option  for  account  POC  to  be  notified  of  all  ECAB  additions,  deletions,  or 
modifications  under  that  account." 

"Provide  option  for  account  POC  to  review  all  ECAB  search  strategy  profiles  under  that 
account." 

"No  longer  able  to  keep  search  terms  and  employment  status  of  ECAB  users  current  as  could 
with  the  paper  CABs.  Annual  review  and  'intervention'  with  end  user  enhanced  user  satisfaction 
&  increased  use  of  the  CAB  service.  Now  seeing  ECAB  usage  declining  because  most  end  users 
don't/won’t  take  time  to  review  &  update  to  maintain  relevancy.” 
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2.  The  2003  overall  DTIC  customer  satisfaction  score  is  down  4  points  from  2002  to  77  percent 
in  2003. 


Possible  explanations  of  why  the  customer  satisfaction  score  went  down  may  be  because  of  the 
many  changes  DTIC  has  experienced  this  past  year.  These  changes  included  a  revised 
registration  process  and  system,  introduction  of  Private  STINET,  and  issues  brought  up  at  the 
users  conference. 


3.  The  number  of  non-deliverable  e-mail  notifications  sent  out  in  the  2003  survey  is  large  (824 
returned  e-mails).  The  following  may  be  possible  reasons. 


•  Server  problems  when  sending  out  initial  survey 

•  Anti-Spam  software  blocking  emails. 

•  E-mail  address  on  file  at  DTIC  not  valid 

•  User  no  longer  at  e-mail  address 

•  Increased  security  due  to  9- 1 1  shut  down  of  agency  e-mail  systems 

•  Server  problems  for  recipient  or  originator 

Recommendation:  Improvements  in  maintenance  of  DTIC  contact  databases  and  registration 
information. 


4.  2003  response  rate  increased  in  comparison  to  last  year. 


An  effort  was  made  by  PROCAP  staff  to  increase  the  response  rate.  The  response  rate  for  the 
2003  CS  survey  increased  12  percentage  points  in  comparison  to  2002  response  rate  which  was 
15  percent.  The  effort  was  two  fold:  (1)  PROCAP  provided  e-mail  maintenance  for  the  CS 
survey  user  contact  list  (i.e.,  search  for  glaring  typos,  research  various  DTIC  databases  for 
verification  or  correct  e-mail  addresses),  and  (2)  Forwarded  two  e-mail  reminders  to  users. 

Recommendations:  Improvements  in  maintenance  of  contact  databases  and  registration 
information.  Continue  to  forward  e-mail  reminders. 


5.  Respondents  to  the  open-ended  questions  provided  several  comments  that  support  the  need 
for  improvements  to  our  Customer  Service  problems  (communication  processes).  The 
comments  below  represent  a  selected  sample  of  quotes  from  users  regarding  this  issue. 


■  "Somewhat  slow  and  lacking  a  single  point  of  contact  to  resolve  issues." 

■  "..  .There  seems  to  be  one  person  for  each  job  function  and  no  one  appears  to  be  cross- 
trained.  As  a  result,  when  a  person  is  out  of  the  office,  there  is  no  one  else  available  to  help 
resolve  issues  in  a  timely  manner." 
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■  "One  thing  I  noticed  for  customer  care  is  that  I  cannot  reach  a  live  person  on  the  phone.  For 
the  most  part,  I  leave  a  message  and  wait  for  a  call  back.  It  would  be  nice  to  be  able  to  reach 
a  person  over  the  phone  the  first  time  instead  of  waiting  for  a  call  back." 

■  "The  people  who  answer  the  phone  (or  who  call  the  customer)  need  to  be  able  to  solve  the 
problem  -  no  matter  what  it  is.  Recommend  much  better  training,  centralized  problem¬ 
solving/  ombudsmen. " 

■  ". .  .No  central  problem  solver  without  going  up  a  couple  of  levels  to  management." 

■  "It  is  sometimes  difficult  to  get  to  the  person  you  need  -  and  when  you  do  -  there  is  not 
always  a  knowledgeable  or  timely  response.  If  you  know  the  right  person  to  call  -  they  can 
usually  help  -  it's  just  getting  to  them.  " 

■  "I  have  not  received  as  timely  responses  as  I  would  have  liked.  Takes  several  days  to  return 
my  telephone  message.  Have  had  trouble  actually  reaching  anyone  by  phone." 

■  "While  I  have  ultimately  received  most  of  the  help  or  assistance  I  have  ever  requested  or 
needed  from  DTIC,  it  has  often  been  a  tedious  affair  to  get  the  required  guidance  or  support." 


Recommendation:  Strong  consideration  should  be  given  to  building  a  call  center  to  solve  some 
of  the  customer  service  problems  mentioned  above. 
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Appendix  A 

DTIC  Customer  Satisfaction  Questionnaire 


DTIC  2003  CUSTOMER  SATISFACTION  SURVEY  QUESTIONNAIRE 


OVERVIEW: 

Please  read  this  carefully  before  starting: 

This  data  collection  is  authorized  by:  DOD  DD-C31  (SA)  2951,  Expires  05/26/04  &  OMB  5CFR  1320.5 
(b)  OMB  0704-0403,  Expires  08/31/04 

This  web-based  assessment  is  voluntary.  However,  your  input  is  essential  toward  improving  our 
continuing  operations  and  helping  us  better  serve  your  information  needs.  The  average  response  time  to 
complete  this  survey  is  about  5  minutes.  Please  read  each  question  carefully  before  responding. 

Thank  you  for  participating. 


SECTION  1:  USER  DEMOGRAPHICS 

1.  Which  of  the  following  best  describes  your  current  organization?  (Click  on  one) 

□  DOD/Military/Civilian  (All  Services/Guard/Reserves/  Labs/Colleges/Schools) 

□  Non-DOD  Federal  Government  (All  Departments/Agencies/Labs) 

□  Industry/Corporate/Govemment  Contractor/Small  Business/  Labs 

□  Historical  Black  Colleges  and  Universities  and  Minority  Institutions  (HBCU/MI) 

□  College/University/Research  Center  (other  than  HBCU/MI) 

□  Other  (please  specify  in  the  space  below) 


Box  to  write  in  answer 


2.  In  general,  your  current  job  position  can  be  best  described  as  which  of  the  following:  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  Contracting  Officer/Procurement  Specialist/COTR/Sponsor 

□  Engineer  (Aeronautical,  Civil,  Industrial,  Mechanical  or  other  Engineering  Fields) 

□  Librarian  (Head,  Technical,  Research  or  other  Library  Staff) 

□  Researcher/Analyst/Program  Manager  (Non-Scientist) 

□  Resource  Management/Financial/Budget  Analyst 

□  Scientist  (Aerospace,  Biology,  Chemistry,  Physics  or  other  Scientific  Field) 

□  Technical  Information  Specialist 

□  Other  (please  specify  in  the  space  below) 


Box  to  write  in  answer 


3.  When  you  acquire  scientific  and  technical  information  from  DTIC,  who  uses  it?  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  I  primarily  provide  information  to  others  (intermediary  user) 

□  I  use  the  information  myself  in  my  current  job  position  (end  user) 
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4.  You  have  been  a  registered  DTIC  user  for:  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  Less  than  1  year 

□  1-2  years 

□  3-5  years 

□  6  or  more  years 


SECTION  2:  INFORMATION  REQUIREMENTS 

5.  Are  you  able  to  find  the  type(s)  of  information  that  you  need  in  the  DTIC  collection?  (Click  on  one) 
Multiple  Choice  (Single  Response) 


□  Yes 

□  Not  Sure 

□  No 

6.  List  your  additional  current  and/or  future  information  needs: 


Box  for  Open  Ended- Write-in 


SECTION  3:  USERS  OF  DTIC  ONLINE  SERVICES 

Please  read  before  continuing  with  the  Survey 

Questions  7  and  8  are  to  be  completed  by  those  individuals  who  currently  use  one  or  more  of  the  DTIC 
online  services.  If  you  do  not  use  any  of  the  online  services  listed ,  skip  to  question  9. 

7.  How  would  you  rate  your  overall  satisfaction  with  DTIC  online  services? 

Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

8.  Which  DTIC  online  service(s)  do  you  currently  use? 


Box  for  Open  Ended- Write-in 
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SECTION  4:  CUSTOMER  SERVICE  EXPERIENCES 


9.  In  your  experience(s)  with  us,  how  would  you  rate  DTIC,  as  a  whole,  on  the  following  customer 
service  quality  factors?  (Click  on  as  many  as  apply) 

Condensed  Question  List: 

□  Accessibility  (Easy  to  access  staff): 

□  Accuracy  (Provided  correct  information): 

□  Courtesy: 

□  Helpfulness: 

□  Knowledge: 

□  Professionalism: 

□  Responsiveness  (Timely  response): 

□  Speed  of  Service  (How  quickly  performed): 

Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

10.  When  you  acquire  documents  from  DTIC,  which  method  do  you  most  often  use?  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  Commercial  Telephone/DSN 

□  Email 

□  Fax 

□  Intemet/Online  Services 

□  Mail 

□  Other  (please  specify  in  the  space  below) 


Box  for  Open  Ended- Write-in 


11.  Y our  comments  about  the  quality  of  DTIC  customer  care: 


Box  for  Open  Ended- Write-in 
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SECTION  5:  COMMUNICATIONS  AND  ACCESS 


12.  Which  communication  channel  do  you  most  often  use  to  contact  DT1C?  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  Commercial  Telephone/DSN 

□  Email 

□  Fax 

□  Intemet/Online  Services 

□  Mail 

□  Other  (please  specify  in  the  space  below) 


Rox  for  Write-in 


13.  DT1C  does  a  good  job  informing  you  about:  (Click  on  all  that  apply) 

Condensed  Question  List: 

□  New  Products/Services 

□  Product/Services  Changes 

□  Submitting  Documents 

□  Training  Opportunities 

□  Ordering  Documents 

□  System  Disruptions/Downtime 

Response 

□  Strongly  Agree 

□  Agree 

□  Disagree 

□  Strongly  Disagree 

□  No  Opinion 

SECTION  6:  GENERAL 

14.  Have  you  used  the  DT1C  Homepage/Web  site  in  the  past  12  months?  (Click  on  one) 

□  Yes 

□  Not  Sure 

□  No 

15.  If  so,  how  would  you  rate  the  following  quality  factors? 

Condensed  Question  List: 

□  Accessibility  (Easy  to  access  staff): 

□  Appearance  (How  does  the  site  look) 

□  Ease  of  Use  (Navigating  the  site): 

□  Organization: 

□  Content: 


Response  Multiple  Choice  ( Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

16.  Have  you  attended  a  DTIC  Annual  Users  Conference  in  the  last  3  years?  (Click  on  one) 

□  Yes 

□  Not  Sure 

□  No 

17.  If  so,  did  you  find  the  conference  valuable  in  terms  of  the  information  provided? 

□  Yes 

□  Not  Sure 

□  No 

18.  Please  include  your  comments  and/or  suggestions  regarding  the  DTIC  Annual  Users  Conference. 

Box  for  Open  Ended  -  Write-in 


19.  How  important  are  DTIC  products/services  to  the  accomplishment  of  your  business  objectives? 

(Click  on  one) 

Response 

□  Extremely  Important 

□  Very  Important 

□  Important 

□  Not  Very  Important 

□  Not  at  All  Important 

20.  Which  of  the  following  describes  DTIC's  impact  in  supporting  your  overall  mission?  (Click  on  one) 
Response 

□  Very  High  Impact 

□  High  Impact 

□  Medium  Impact 

□  Low  Impact 

□  Very  Low  Impact 

21.  Would  you  recommend  DTIC  to  a  friend  or  colleague?  (Click  on  one) 

Multiple  Choice  (Single  Response) 

□  Yes 

□  Not  Sure 

□  No 
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22.  Please  include  your  comments  and  suggestions  for  improving  DTIC  products,  services  and/or  our 
customer  care. 


Box  for  Open  Ended  -  Write-in 


NOTE: 

Thank  you  for  participating  in  this  important  survey. 


|SUMMIT|  |CLEAR| 
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Appendix  B 
Quantitative  Results 


Reference  Fig  2. 1-2.3 


2003  CSS  Overall  Cust  Satis 


Caue  tUon  i 

Count 
(  Valid  N) 

Mean 

(norm.) 

Mean  at  a  percent otponlble 
•core 

O  20  «0  60  so  too 

Favorable 

Neutral 

Unfa  vorable 

Accessibility  (Easy  to  access  staff): 

669 

72.25 

70.4% 

23.3% 

6.3% 

Accuracy  (Provided  correct  information): 

662 

76.25 

77.2% 

19.5% 

3.3% 

Courtesy: 

648 

81.50 

84.0% 

14.8% 

1.2% 

Helpfulness: 

651 

79.00 

79.3% 

17.8% 

2.9% 

Knowledge: 

646 

78.00 

79.4% 

18.3% 

2.3% 

Professionalism: 

648 

80.75 

82.7% 

15.6% 

1.7% 

Responsiveness  (Timely  response): 

653 

75.25 

74.4% 

20.2% 

5.4% 

Speed  of  Service  (How  quickly  performed): 

643 

73.00 

70.3% 

24.1  % 

5.6% 

Overall  Averages 

652.5 

77.00 

77.2% 

19.2% 

3.6% 

Reference  Fig  2.4 


How  important  are  DTIC  products/services  to  the  accomplishment  of  your 
business  objectives?  (Click  on  one) 


Response 

Count 

Percent 

0  20  4 

0  6 

0  8 

8  100 

Extremely  Important 

150 

21 .6% 

Very  Important 

184 

26.4% 

m 

Important 

257 

36.9% 

H 

Not  Very  Important 

Not  at  All  Important 

91 

14 

13.1% 

2.0% 

Reference  Fig  2.5 

Which  of  the  following  describes  DTIC's  impact  in  supporting  your  overall 
mission?  (Click  on  one) 


Reference  Fig  2.6 

Would  you  recommend  DTIC  to  a  friend  or  colleague?  (Click  on  one) 


Response 

Count 

Percent 

0  20  40  60  80  100 

Yes 

623 

88.6% 

Not  Sure 

68 

9.7% 

No 

12 

1 .7% 
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Reference  Fig  3.1 


DTIC  Online  Service  Overall  Satisfaction 


Queitloni 

Count 
•  Valid  N) 

Mean 

(norm.) 

Mean  at  a  percent otponlble 
•core 

O  20  40  60  80  ICO 

Favorable 

Neutral 

Unfavorable 

How  would  you  rate  your  overall 
satisfaction  with  DTIC  online  services? 

570 

69.00 

64.9  % 

28.1% 

7.0% 

Overall  .Averages 

570.0 

69.00 

64.9 % 

28.1% 

7.0% 

Reference  Fig  3.2  Librarians 


Quetlloni 

Count 
•  valid  N) 

Mean 

(norm.) 

Mean  at  a  percent  otponlble 
•core 

O  20  40  60  80  too 

Fa  vorable 

Neutral 

Unfa  vorable 

How  would  you  rate  your  overall 
satisfaction  with  DTIC  online  services? 

158 

68.50 

64.6% 

28.5% 

7.0% 

Overall  Averages 

158.0 

68.50 

64.6% 

28.5% 

7.0% 

Reference  Fig  3.2  Engineers 


Queitloni 

Count 
•  valid  M  i 

Mean 

(norm.) 

Mean  at  a  percent  otponlble 
•core 

O  20  40  60  80  100 

Favorable 

Neutral 

Unfavorable 

How  would  you  rate  your  overall 
satisfaction  with  DTIC  online  services? 

106 

68.50 

62.3% 

29.2% 

8.5% 

Overall  Averages 

106.0 

68.50 

62.3% 

29.2% 

8.5% 

Reference  Fig  3.2  Researcher/ Analyst 


Queitloni 

Count 
•  valid  N) 

Mean 

(norm.) 

Mean  ai  a  percent  otponlble 
•core 

0  20  40  60  80  too 

Favorable 

Neutral 

Unfa  vorable 

How  would  you  rate  your  overall 
satisfaction  with  DTIC  online  services? 

84 

68.25 

66.7% 

23.8% 

9.5% 

Overall  Averages 

84.0 

68.25 

66.7% 

23.8% 

9.5% 

Reference  3.3 

Have  you  used  the  DTIC  Homepage/Web  site  in  the  past  12  months?  (Click  on 
one) 
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Reference  Fig  3.4 

2003  General  Overall  Homepage  Oust  Sat. 


Cue ilon i 

Count 
•  Valid  N) 

Mean 
•  norm.) 

Mean  a«  a  percent  or  ponlble 
•core 

O  20  40  60  80  100 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Easy  to  access  site): 

565 

74.00 

73.5% 

21.8% 

4.8% 

.Appearance  (Hour  does  the  site  look): 

562 

71.50 

68.0% 

27.0% 

5.0% 

Ease  of  use  (Navigating  the  site): 

560 

65.00 

58.8% 

25.7% 

15.5% 

Organization: 

556 

66.50 

60.3% 

26.1% 

13.7% 

Content: 

558 

70.25 

67.6% 

24.7% 

7.7% 

Overall  Averages 

560.2 

69.45 

65.6% 

25.1% 

9.3% 

Reference  3.5-3.12 


Homepage  Performance  Rating  -  Librarian 


Oue ilon i 

Count 
•  Valid  N) 

Mean 
i  norm . ) 

Mean  at  a  percent  of  ponlble 
•core 

O  20  40  60  80  100 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Easy  to  access  site): 

139 

72.75 

73.4% 

20.1% 

6.5% 

Appearance  (Hour  does  the  site  look): 

138 

73.75 

77.6% 

18.8% 

3.6% 

&se  of  use  (Navigating  the  site): 

136 

67.00 

68.4% 

17.6% 

14.0% 

Organization: 

135 

68.25 

68.1% 

20.0% 

11.9% 

Content: 

137 

72.00 

75.2% 

20.4% 

4.4% 

Overall  Averages 

137.0 

70.75 

72.5% 

19.4% 

8.1% 

Reference  3.5-3.12 


Hompage  Performance  Rating-  Engineers 


Cue ilon i 

Count 
•  valid  N) 

Mean 

(norm.) 

Mean  at  a  percent  or  ponlble 
•core 

O  20  40  60  80  100 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Easy  to  access  site): 

104 

74.75 

74.0% 

21.2% 

4.8% 

Appearance  (Hour  does  the  site  look): 

102 

69.75 

60.8% 

33.3% 

5.9% 

Ease  of  use  (Navigating  the  site): 

102 

64.50 

56.9% 

27.5% 

15.7% 

Organization: 

102 

66.50 

58.8% 

27.5% 

13.7% 

Content: 

102 

70.00 

65.7% 

28.4% 

5.9% 

Overall  Averages 

102.4 

69.10 

63.2% 

27.6% 

9.2% 
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Reference  3.5-3.12 


Homepage  Performance  -  Researcher/Analyst 


Queillont 

Count 
(Valid  H  • 

Mean 

(norm.) 

Mean  at  a  percent  of  possible 
•core 

0  20  40  60  80  ICO 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Easy  to  access  site): 

91 

75.00 

71.4% 

26.4% 

2.2% 

Appearance  (How  does  the  site  look): 

91 

71.00 

62. 6* 

33.0% 

4.4% 

Ease  of  use  (Navigating  the  site): 

91 

64.00 

52.7% 

33.0% 

14.3% 

Organization: 

91 

63.75 

53.8% 

30.8% 

15.4% 

Content: 

91 

69.00 

60.4% 

33.0% 

6.6% 

Overall  Averages 

91.0 

68.55 

60.2% 

31.2% 

8.6% 

Reference  3.13 


Have  you  attended  a  DTIC  Annual  Users  Conference  in  the  last  3  years?  (Click  on 
one) 


Response 

Count 

Percent 

0  2 

0  4 

0  6 

0  8 

0  100 

Yes 

Not  Sure 

No 

82 

5 

611 

1 1 .7% 

0.7% 

87.5% 

Reference  4.1 


Which  of  the  following  best  describes  your  current  organization?  (Click  on  one) 


Response 

Count 

Percent 

0  20  40  6 

0  8 

0  100 

D  o  D/M  i  1  it  a  ry/C  ivi  1  i  a  n  (All 

387 

53.2% 

Non-DoD  Federal  Government  (All 

30 

4.1% 

Industry/Corporate/Government 

239 

32.9% 

Historically  Black  Colleges  and 
College/University/Research  Center 
Other  (please  specify  in  the  space 

17 

33 

21 

2.3% 

4.5% 

2.9% 

1 
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Reference  4.2 


In  general,  which  of  the  following  best  describes  your  current  job  position?  (Click 
on  one) 


Response 

Count 

Percenl 

Contracting  Officer/Procurement 

18 

2.5% 

Engineer  (Aeronautical,  Civil, 

125 

17.4% 

Librarian  (Head,  Technical,  Research 

174 

24.2% 

Researcher/Analyst/Program  Manager 

111 

15.5% 

Resource 

20 

2.8% 

Scientist  (Aerospace,  Biology, 

79 

1 1 .0% 

Technical  Information  Specialist 

69 

9.6% 

Other  (please  specify  in  the  space 

122 

17.0% 

20 


40 


60 


80  100 


Reference  4.3 

When  you  acquire  scientific  and  technical  information  from  DTIC,  who  uses  it? 
(Click  on  one) 


Response 

Count 

Percent 

0  20  46  6 

0  8 

0  100 

1  primarily  provide  information  to 

1  use  the  information  myself  in  my 

394 

321 

55.1% 

44.9% 

Reference  4.4 

You  have  been  a  registered  DTIC  user  for:  (Click  on  one) 


Are  you  able  to  find  the  type(s)  of  information  that  you  need  in  the  DTIC 
collection?  (Click  on  one) 
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Reference  5.1 


Which  communication  channel  do  you  most  often  use  to  contact  DTIC?  (Click  on 
one) 


Response 

Count 

Percent 

0  20  4 

0  6 

0  8 

0  100 

Commercial  Telephone/DSN 

193 

28.2% 

Email 

165 

24.1% 

Fax 

6 

0.9% 

Internet/Online  Services 

301 

43.9% 

Mail 

Other  (please  specify  in  the  space 

6 

14 

0.9% 

2.0% 

Reference  5.2 

DTIC  does  a  good  job  informing  you  about:  (Click  on  all  that  apply) 


New  Products/Services 


Response 

Count 

Percent 

0  20  4 

0  6 

0  8 

0  100 

Strongly  Agree 

Agree 

Disagree 

Strongly  Disagree 

No  Opinion 

156 

385 

27 

9 

90 

23.4% 

57.7% 

4.0% 

1.3% 

13.5% 

: 

L. 

Product/Services  Changes 

Response 

Count 

Percent 

0  20  4 

0  6 

0  8 

0  100 

Strongly  Agree 

Agree 

Disagree 

Strongly  Disagree 

No  Opinion 

163 

384 

30 

9 

81 

24.4% 

57.6% 

4.5% 

1.3% 

12.1% 

: 

i. 
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Submitting  Documents 


Response 

Count 

Pei  cent 

0  2 

0  4 

0  6 

0  8 

0  100 

Strongly  Agree 

Agree 

Disagree 

Strongly  Disagree 

No  Opinion 

105 

308 

40 

12 

191 

16.0% 

47.0% 

6.1% 

1.8% 

29.1% 

: 

- 

. 

Training  Opportunities 


Response 

Count 

Percent 

0  2 

0  4 

0  6 

0  8 

0  100 

Strongly  Agree 

117 

17.8% 

Agree 

324 

49.3% 

Disagree 

53 

8.1% 

■ 

Strongly  Disagree 

16 

2.4% 

No  Opinion 

147 

22.4% 

1 

Ordering  Documents 


Response 

Count 

Percent 

Strongly  Agree 

125 

19.0% 

Agree 

370 

56.2% 

Disagree 

40 

6.1% 

Strongly  Disagree 

11 

1.7% 

No  Opinion 

112 

17.0% 

20 


40 


60 


80  100 


System  Disruptions/Downtime 


Response 

Count 

Percent 

Strongly  Agree 

205 

31 .3% 

Agree 

292 

44.6% 

Disagree 

37 

5.6% 

Strongly  Disagree 

7 

1.1% 

No  Opinion 

114 

17.4% 

3 


60 


80  100 
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Reference  5.3 


When  you  acquire  documents  from  DTIC,  which  method  do  you  most  often  use? 
(Click  on  one) 


Response 

Count 

Percent 

0  2 

0  4 

0  6 

0  8 

0  100 

Commercial  Telephone/DSN 

61 

8.9% 

Email 

87 

12.6% 

Fax 

13 

1.9% 

Internet/Online  Services 

406 

59.0% 

Mail 

91 

13.2% 

Other  (please  specify  in  the  space 

30 

4.4% 
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